A Fast Randomized Incremental Gradient Method for Decentralized Nonconvex Optimization

In this article, we study decentralized nonconvex finite-sum minimization problems described over a network of nodes, where each node possesses local batch data samples. context, analyze single-timescale randomized incremental gradient method, called GT-SAGA . is computationally efficient as it evaluates one component per iteration and achieves provably fast robust performance by leveraging node-level variance reduction network-level tracking. For general smooth problems, show the almost sure mean-squared convergence to first-order stationary point further describe regimes practical significance, outperforms existing approaches topology-independent complexity, respectively. When global function satisfies Polyak–Łojaciewisz condition, that exhibits linear an optimal solution in expectation interest topology independent improves upon methods. Numerical experiments are included highlight main aspects settings.